Experimenal Methdology for Evaluating Learning in Serious Games

This paper reports on a project to evaluate the effectiveness of using a Serious Game as a learning tool. A “serious game” has an explicit and carefully thought-out educational purpose and is not intended to be played primarily for amusement. The formal experimental method was designed to compare formal learning of the A* pathfinding algorithm with a digital serious game implementation that utilized immediate feedback as a mechanism to promote learning. A qualitative analysis of users’ experience of the serious game was highly positive and the feedback component rated it as highly relevant for learning. However, an initial attempt to quantitatively evaluate learning in a serious game through the formal experiment did not achieve the expected results. This paper discusses useful insights gained and lessons learned from the implementation. It reflects on the possible contradiction between evaluation and games, where games often encourage making mistakes as a mechanism for learning while evaluation typically regards error as failure. Future development of serious games and refining the learning evaluation methodology are proposed in light of the lessons learned.